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Abstract 

Online education is rapidly becoming a significant method of course delivery in higher education. 
Consequently, instructors analyze student perfonnance in an attempt to better scaffold student 
learning. Learning analytics can provide insight into online students’ course behaviors. Archival 
data from 167 graduate level education students enrolled in 4 different programs and 9 different 
online courses were analyzed to determine whether a relationship existed between grades earned 
and time spent in specific areas within the course: total course time, course modules, document 
repository, and synchronous online sessions. Time spent in each component did not predict a 
specific letter grade, but did predict whether or not an A would be achieved. The sample was 
composed of students from four different graduate education programs: Educational Leadership, 
Reading, Instructional Design, and Special Education. Variations found among programs did not 
significantly predict the grade earned in the course. A logistic regression revealed that of all the 
predictor variables, time spent in synchronous online sessions alone showed as a significant 
predictor of receiving an A in the course. This is important infonnation for instructors when 
providing scaffolding for students. 
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Relationship Between Grades Earned and Time in Online Courses 
Growth in Online Courses 

Online education plays a significant role in higher education, especially among adult 
learners atthe university level (Jo, Kim, & Yoon, 2015). The majority of American college courses 
have some digital components, ranging from the fully online to Web enhanced. Consequently, 
digital learning has become an integral part of instructional delivery in much of higher education 
(Ciabocchi, Ginsberg, & Picciano, 2016). 

As of 2016, Allen and Seaman (2016) reported that there were between 5.5 and 7 million 
higher education students enrolled in at least one online course. This was more than a 100% 
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increase from the total just six years previously (Allen & Seaman, 2010). Although both graduate 
and undergraduate courses are offered online, graduate students have found this format to be 
particularly advantageous because of reduced time constraints allowing adult learners a more 
flexible schedule (Putman, Ford, & Tancock, 2012). 

Studies comparing online learning and face-to-face instruction revealed that research since 
1998 had recorded better learning outcomes from online courses than from face-to-face courses 
(Zhao, Lei, Yan, Lai, & Tan, 2005). Zhao et al.’s findings suggested that the technological 
advances which enhanced two-way interaction might promote this online advantage. Means, 
Toyama, Murphy, Bakia, and Jones (2009), in their analysis of 56 rigorous studies of online 
education, found that learners in online settings significantly outperformed their peers in face-to- 
face settings in the use of metacognitive strategies. These skills included self-reflection, self¬ 
explanation, and self-monitoring. 

Expanded technological support and enhanced online learning experiences offered through 
learning tools within Learning Management Systems (LMS) have significantly improved learning 
in online courses. Since the late 1990s, approximately 93% of surveyed U.S. higher education 
institutions (Campus Computing, 2010) report having adopted web-based LMS to deliver online 
learning courses. In another study, almost 94% of survey participants from 417 colleges and 
universities stated that “digital curricular resources make learning more efficient and effective for 
students” (Campus Computing, 2015, para. 1) and 96% of surveyed institutions agreed that 
“adaptive learning technology has great potential to improve learning outcomes” (Campus 
Computing, 2015, para. 1). 

Student Performance in Online Courses 

Even though performance and use of metacognitive strategies are effective in the online 
environment, online learners tend to face additional challenges. Without the support of a structured 
classroom environment, online learners frequently face time management issues caused by 
juggling the demands of course work, employment, and other responsibilities despite the presence 
of learning and collaborative tools. Consequently, successful completion of online course work 
can be impacted by learners’ time management skills (Joo, Jang, & Lee, 2007). Another 
component that could affect perfonnance in online learning is the sense of community. Course 
components such as real-time lectures and group discussions using web conferencing tools might 
enhance online course experiences. Real time student-to-student and student-to-teacher 
interactions (Falloon, 2011; Pattillo, 2007) help build an important sense of community 
(Hrastinski, 2008). The extent to which students are using the various tools provided for them in 
the online environment can be examined using learner analytics data that are available within the 
LMS. This information can guide professors to understand typical patterns of student usage and 
their correlation to student grades. However, the usefulness of learning analytics to an institution 
depends upon data availability through their LMS and instructors’ ability to analyze the data. This 
analytic information varies and is not necessarily incorporated into a coherent, widely understood 
plan or strategy for data use and interpretation (West & Heath, 2016). 

Analytics: A data mining tool 

Learning analytics is a tool embedded in a Learning Management System for “the 
measurement, collection, analysis, and reporting of data about learners and their contexts, for 
purposes of understanding and optimizing learning and the environments in which it occurs” (Long 
& Siemens, 2011, p. 32). This tool, according to Campbell, DeBlois, and Oblinger (2017), produce 
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“actionable intelligence” for use by the institution and is therefore frequently used for enrollment 
and other administrative purposes; however Macfadyen and Dawson (2012) and Picciano (2012) 
determined that analytics can provide a detailed understanding of how students spent time in LMS- 
supported courses related to the usage of the tools and pedagogical strategies employed by the 
learners and how the data can be used to fonn conclusions to improve learning. The data provide 
infonnation about specific variables that enhance educational opportunities, thereby allowing 
instructors to intervene when students are at risk or to provide additional feedback and instructional 
content when appropriate (Avella, Kebritchi, Nunn, & Kanai, 2016). 

Because online course format can result in isolated learners, research has emphasized the 
importance of developing peer interaction to facilitate learning (Tinto, 1998). In order to examine 
peer to peer interaction, Dawson et al. (2008) analyzed the types and frequency of interactive tools 
used within the LMS. They found that measuring the average time that a learner spends online 
using an LMS provided merely “a crude indicator of student time investment in learning” 
(Macfadyen & Dawson, 2012, p. 153). Student engagement has been examined in many studies. 
Graduate and undergrad students have been found to have differing reactions to the importance of 
various components within online courses. Schroeder, Baker, Terras, Mahar, and Chiasson (2016) 
noted that graduate students desired connectivity with their instructors but connectivity with other 
students was not viewed as important, while undergrad students viewed both as important. Carver 
et al. (2013) noted that the most important factor in using synchronous sessions for graduate 
students was not simply adding the sessions to the course, but to specifically use the sessions to 
support and expand course content. The difference in perspective between graduate and 
undergraduate students might be a significant contributing factor affecting voluntary attendance at 
synchronous sessions within this study. 

To further analyze what students were doing while logged into LMS-based course sites, 
data from the LMS tool usage can be analyzed. Macfadyen and Dawson (2012) divided the online 
tools into four broad categories: Engagement with Learning Community, Working with Content, 
Assessment, and Administrative Tasks. Their sample of almost 4,000 graduate and undergraduate 
course sections indicated that students used tools from the Working with Content category than 
they did in the other categories, which resulted in a significant correlation between student use of 
tools with the Working with Content category and students’ final course grade. However, they also 
found a significant positive correlation between increased use of tools in the Engagement with 
Learning Community category and academic success (Macfadyen & Dawson, 2010). They found 
a significant positive correlation between students’ final grade in the course and their use of the 
tools within the Engagement with Learning Community category. This category included use of 
the discussion boards and the course email (Macfadyen & Dawson, 2010). Previously, Dawson, 
McWilliam, and Tan (2008) had only identified a significant positive correlation with first year 
college science students between final grades and students’ use of course content materials. The 
research questions in our study attempted to detennine whether the same conclusion applied to 
graduate students. 

Research has determined that learning analytics can be used to help instructors determine 
student learning outcomes as well as to detennine how to improve student’s academic performance 
(Bhardwaj & Pal, 2011). In an attempt to further analyze what student were doing while logged 
into LMS-based course sites, the current research study examined data on LMS tool usage. In the 
current research study, the researchers examined the online course analytics from graduate level 
courses in the Department of Education at a four-year private university. The areas analyzed 
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included: total time in course, time spent in the content module, the document repository, and the 
online synchronous sessions. This study examined two of Dawson et al.’s (2008) four areas: 
Working with Content (content modules and document repository) and Engagement (ClassLive) 
more extensively to determine the effects of usage of these tools on final course grade. Based on 
increasing evidence that student engagement with peers in a learning community has a strong 
positive correlation with learning success, the researchers wanted to specifically examine these 
two categories. It was hypothesized that there would be a significant correlation between the 
various LMS elements, amount of time spent within the online course the modules, the document 
repository, and the synchronous online tool, and whether a graduate student earned an A in the 
course (Macfadyen & Dawson, 2010). 

Research Questions 

For this study, the researchers used the following research question to guide the study: 
What relationship exists between the total amount of time graduate students spend in the various 
online course elements—total time within course, course modules, document repository, and 
synchronous online tool—and whether the student earned an A in the course? 

Hof. In an online course, there will be not be a relationship between total time spent in the 
course, in the course modules, synchronous online class sessions, or the document 
repository and the grade earned in the course. 

Hai'. In an online course, there will be a significant relationship (p < .05) between total time 
spent in the course, in the course modules, synchronous online class sessions, or the 
document repository and the grade earned in the course. 

Hq 2 '. In an online course, there will be not be a relationship between total time spent in the 
course, the course modules, synchronous online class sessions, or document repository and 
earning an A in the course. 

Hai'. In an online course, there will be a significant relationship (p < .05) between total time 
in course modules, synchronous online class sessions, or document repository and earning 
an A in the course. 

Hop. In an online course, there will not be a significant relationship (p < .05) between the 
program and the time spent in the course modules, synchronous online class sessions, or 
the document repository and the grade earned in the course. 

Ha3'. In an online course, there will be a relationship (p < .05) between the program and the 
time spent in the course modules, synchronous online class sessions, or the document 
repository and the grade earned in the course. 


Methods 

Learning analytics archival data from students enrolled in courses in four graduate 
programs in Education: Reading, Exceptional Education, Educational Leadership, and 
Instructional Design at a small southeastern university were analyzed to determine the relationship 
between the time students spent in various course elements and their final course grade. 

Data from the LMS from 167 Master’s level students enrolled in 10 course sections during 
the fall 2014 and the spring 2015 were analyzed (See Table 1). The sample included 47 students 
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from the Reading program, 59 students were from the Educational Leadership program, 36 
students were from the Exceptional Student Education program, and 25 students were from the 
Instructional Design program. To correct for variation among instructors, the sample included 
courses taught by seven different graduate education professors. However all of the sections were 
taught using master course syllabi and master course shells designed to incorporate the same 
elements. Each course, therefore, included eight common elements: content modules, a document 
repository for assignments and course related infonnation, weekly discussion boards for student 
interactions related to the content, and weekly synchronous online class hours. The graduate 
students earned grades based upon the quality of the discussion postings, while no grade was 
attached to the modules, document repository, or synchronous components. The researchers 
acknowledge that online learners are limited in their ability to avail themselves of synchronous 
sessions because of constraints of time and geographical location; hence no part of the course grade 
was tied to participation in the synchronous sessions. Additionally, recorded sessions of the live 
sessions were made available to students immediately after the session for students to access at 
their convenience. 

As these are graduate level courses, the numbers of students earning Cs, Ds, or Fs were 
negligible. A total of 123 students earned As, 34 students earned Bs, 7 students earned Cs, 2 
students earned Ds and 1 students earned an F. The independent variables identified for the analysis 
were total time spent by students within the course LMS, time spent in the modules within the 
online classroom, time spent in the document repository area, and time spent in synchronous online 
sessions. The dependent variable used in the analysis was the course grade earned by the student. 
The demographic data by program and independent variable are provided in Table 1. 



Educational 

Instructional 

Reading 

Special 

Total 


Leadership 

Design 


Education 



(N=123) 

(N=25) 

(N=47) 

(N=36) 

(N=167) 

Total time within the 

90.1 hours 

84.8 hours 

75.8 hours 

96.7 hours 

86.7 hours 

Course 

(SD=46.2) 

(SD=46.8) 

(SD=46.8) 

(SD =44.9) 

(SD =43.9) 

Time spent in Synchronous 

2.3 hours 

2.0 hours 

2.9 hours 

2.4 hours 

2.5 hours 

Online Sessions 

(SD = 2.5) 

(SD = 1.6) 

(SD = 2.4) 

(SD = 1.9) 

(SD = 2.3) 

Time spent in the Modules 

35 hours 

40.5 hours 

35.9 hours 

46.6 hours 

38.5 hours 


(SD = 17.6) 

(SD = 22.5) 

(SD = 15.9) 

(SD = 24.1) 

(SD = 19.9) 

Time spent in Document 

3.9 hours 

3.6 hours 

7.7 hours 

4.6 hours 

5.1 hours 

Repository 

(SD = 3.8) 

(SD = 2.5) 

(SD = 3.8) 

(SD = 3.3) 

(SD = 3.9) 

Percent Earning an A 

83.1% 

72.0% 

76.6% 

55.6% 

73.7% 


Table 1. Demographic Information 


A one-way ANOVA was used to analyze significant differences in time spent within the 
portions of the course and the student’s final grade. The ANOVA revealed that there was a 
significant difference based on the total time within the course (F( 2, 164) = 6.93,/? < .001), time 
spent in the synchronous online sessions (F(2,164) = 7.15, p < .001), and time spent within 
modules (F(2,164) = 6.65, p = .002) (Table 2). However, there were not significant differences 
based on time spend in document sharing, (F(2,164) = 1.08, p =.342). 

Post-hoc analysis showed statistically significant differences between those students 
earning an A and those earning any other grade. However, no difference was found between 
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students earning B, C, or below. It can be concluded, therefore, that time invested in the LMS did 
not seem to help distinguish between perfonnances at different grade levels below an A. As a 
result, the data were merged into two groups: those students earning an A and those students who 
did not earn an A. T-tests were then used to confirm whether there was a difference between the 
mean time spent in each area when compared to those students who earned an A and those who 
did not earn an A. The difference between the mean time spent in the document repository of those 
earning an A and those not earning an A was not significant. 



Earning an A 
(N=123) 

Not Earning an A 
(N=44) 

P Value 

Total time within the Course 

93 hours 
(SD=46.1) 

67.2 hours 
(SD =30.2) 

.001* 

Time spent in Synchronous Online 
Sessions 

2.8 hours 
(SD = 2.3) 

1.4 hours 
(SD= 1.8) 

.001* 

Time spent in the Modules 

41.5 hours 
(SD = 20.7) 

24.7 hours 
(SD = 14.9) 

.002* 

Time spent in Document Repository 

5.25 hours 
(SD = 3.8) 

4.68 hours 
(SD = 4.3) 

.342 


P* denotes probability 

Table 2. Results comparing mean times between earning an A and not earning an A 


Since the analytics data were analyzed and compared from 4 different programs, the 
researchers also wanted to determine whether the program (Reading, Exceptional Student 
Education, Educational Leadership, or Instructional Design) impacted the results. An ANOVA 
was performed which revealed differences between the program the student was enrolled in, the 
amount of time spent in the document repository (F(3,163) = 11.91 ,P< .001), and time spent in 
the 8 content modules (F(3,163) = 3.046 ,p = .030). Sheffe’s post-hoc follow up analysis showed 
that students in Educational Leadership spent more time in the modules than Reading students, 
while Reading students spent more time in document sharing than students in all other programs. 
There seem to be slight differences in the ways each program used the various components of the 
LMS. Additionally, it was found that grade distribution also differed by program with 83.1% of 
Educational Leadership students receiving an A, followed by Reading (76.6%), Instructional 
Design (72.0%), and Special Education (55.6%), % 2 (3,167) = 9.01 ,p =.029. In addition to time spent 
in modules and student grade distribution, program enrollment was included in further analysis in 
order to determine the effect of the synchronous online session time and time spent in content 
modules above and beyond the effect of program type. 

A logistic regression was conducted to predict the probability of receiving an A in a 
Masters level education course (Table 3). In Model 1, program type was entered as a covariate to 
control for the effect of the program in which students were enrolled in order to get a baseline 
model. The logistic regression used in Model 2 included the predictor variables of total time in 
course, synchronous online session time, time spent in course modules, and time spent in the 
document repository. Finally, Model 3 included both the covariate and predictor variables, which 
significantly predicted whether students received an A or a different grade in the course, % 2 (3,167) 
= 36.86, p < .001. Of all the predictor variables, only time spent in synchronous online sessions 
showed as a significant predictor of receiving an A. 
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Predictor 

fi(SE) 

Model 1 

Odds 

Ratio 

95% Cl 

Model 2 

Odds 

fi (SE) Ratio 95% Cl 

Model 3 

Odds 

fi (SE) Ratio 

95% Cl 

Intercept 

.223 

1.25 


-.495 (.48) 



-2.18 (.71) 

.114 



(.34) 









Program 










ID 

1.366 

3.92 

1.52- 




2.057 (.58)** 

7.83 



(.48)** 


10.09 







Reading 

0.721 

2.06 

0.69- 




1.204 (.64) 

3.33 



(.56) 


6.14 







Special Ed 

0.962 

2.62 

1.02- 




1.724 (.61)* 

5.61 



(.48)* 


6.72 







Course Total 




.009 (.01) 

1.01 

0.99- 

.005 (.01) 

1.01 

0.98- 







1.03 



1.03 

Synchronous 




.296 (.12)* 

1.34 

1.06- 

.339 (.13)* 

1.40 

1.08- 

online 






1.70 



1.82 

sessions 










Module 




.015 (.02) 

1.02 

0.97- 

.036 (.03) 

1.04 

0.99- 







1.06 



1.09 

Document 




-.056 (.05) 

0.95 

0.85- 

-.093 (.07) 

0.91 

0.80- 







1.05 



1.04 


*p < .05, **p < .001 

Model 1: % 2 (3,167) = 8.65 ,p < .05, R 2 = .074 (Nagelkerke) 
Model 2: X 2 (4,167) = 21.52, p < .001, R 2 = All (Nagelkerke) 
Model 3: : X 2 (7,167) = 36.86,< .001, R 2 = .289 (Nagelkerke) 

Table 3. Logistic Regression 


Conclusion 

Time invested in an online course is only one variable that can impact students’ success. 
The results of this study supported Mcfadyen and Dawson’s (2010) findings that time spent on the 
tools within an online course does not necessarily correlate with grades earned. Across all 4 
programs, the mean time spent in 8-week graduate courses was 86.7 hours with the majority of 
this time being spent within the course modules. However, it did not follow that the greater the 
amount of time spent within the course, the higher the grade earned. The data did not help to 
distinguish between the students earning a grade of A and those students earning less than an A. 
Consequently, the null hypothesis was accepted for our first research question. 

Hoi: In an online course, there will be not be a relationship between total time spent in the 
course, in the course modules, synchronous online class sessions, or the document 
repository and the grade earned in the course. 

An examination of the analytic data revealed that there was a difference in the amount of 
time invested by those who earned an A and those who did not earn an A. A statistically significant 
relationship existed among three of the components we examined. Total time in course, time spent 
in the content modules, and time within synchronous online sessions all had a significant 
correlation to earning an A as compared to not earning an A in the course. However, time spent in 
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the document repository was not significant. Consequently, we rejected the second null hypothesis 
and accepted the alternate second hypothesis. 

Ha:'■ In an online course, there will be a significant relationship (p < .05) between total time 
in the course, in the course modules, synchronous online class sessions, or document 
repository and earning an A in the course. 

Although there was a statistically significant relationship, using a logistic regression was 
particularly informative because it revealed that only time spent in synchronous online sessions 
appeared to predict whether a student would earn an A in the course. For every 20 minutes spent 
in a synchronous online session, the likelihood of earning an A increased by 1.4 times. This seems 
to indicate the value of providing synchronous teacher-student and student-student interaction, 
which supports Tinto’s (1998) observation about the importance of peer-to-peer interaction. It is 
to be noted that attendance at the synchronous sessions was voluntary and grades were not 
associated with attendance. 

Another observation from the analytics data was a slight difference in students’ 
performances between programs. Out of the four programs, students in the Educational Leadership 
students were more likely to earn an A, while Special Education students were least likely to earn 
an A. This information cannot be generalized to determine if the variance is impacted by 
characteristics of students admitted to the program, or by variables in the construction of the online 
courses. This type of a study cannot be used to determine causality. 

There were also programmatic differences between the means of time spent in each portion 
of the LMS. The mean differences by program between the time spent in various parts of the LMS 
might be indicative of variations in the ways that the portions of the courses were constructed by 
program. However, differences between programs were not as significant as the difference 
between those students earning an A and those not earning an A. 

Has- In an online course, there will be a relationship (p < .05) between the program and the 
time spent in the course modules, synchronous online class sessions, or the document 
repository and the grade earned in the course. 

The logistic regression provided the most important information. This information could 
be used to help instructors support student performance. For every 20 minutes (.339 units of an 
hour) students spent in synchronous online sessions, it increased the likelihood of receiving an A 
by 1.4 times, regardless of graduate program. Instructors who are aware of this figure would be 
able to explain the benefits of the synchronous sessions to the students and thereby encourage 
student participation. 

As the LMS analytics become even more sensitive, this increased information would allow 
researchers and instructors to examine online student behavior even more specifically. Improved 
analytic information could provide additional data to more proactively support students during 
online courses, rather than simply analyzing student activity after the course completion. 

Limitations and Further Study 

The results of the research indicate that student participation in various portions of the LMS 
as measured by the learning analytics has a significant positive relationship with student 
achievement in graduate level online courses. The data indicates that an increase in student 
participation, especially in online synchronous sessions, should translate into an increase in student 
achievement in graduate online courses. However, the findings of this study have limitations. As 


Online Learning Journal - Volume 21 Issue 4 - December 2017 


310 



Relationship Between Grades Earned and Time in Online Courses 


is true of any cross-sectional study, the results of this research establish the strength of the 
relationship between the variables but cannot prove that one variable is the cause of the change in 
the other variable. Thus, the study is limited by only being able to show an inter-variable, rather 
than causal, relationship. Additionally, time spent in the course, modules, or synchronous online 
learning only provide information about amount of time logged into each activity, not what 
occurred during that time. Further studies should look at other measures of engagement which 
capture involvement in learning activities, which could help to identify which aspects of student 
activity during logged in time are most beneficial to students. Finally, additional, confounding 
variables may impact the relationship among variables under study (Mitchell, 1985). 

This study raises further questions about the exact factors that make the synchronous online 
sessions within the courses so significant. Further study of the components of the synchronous 
sessions and how they are used within each of the programs would provide instructors with 
valuable information for better scaffolding the student learning experience. The results of this 
study cannot be generalized to other populations outside of the online graduate education students. 
The findings from this study might not be applicable to undergraduate students or those students 
enrolled in blended courses. 
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